Test collections for all

Researchers working in the IR field have placed much reliance on building test collections that can be used widely by many researchers. Many collections have been used for years even decades. In the age of contextual IR, this talk will advocate an alternative less tried approach, that of building many context specific collections, that don’t require a great deal of effort to build but may not be all that re-usable. There is an increasing quantity of research examining this area and through an overview of this research, I shall argue that building your own collection is a better approach to take. Bio Dr. Mark Sanderson is an associate professor at the University of Sheffield. He has published extensively in the filed of Information Retrieval producing a number of highly cited papers including a number in the field of evaluation. With Paul Clough, Dr. Sanderson initiated the successful ImageCLEF evaluation exercise and more recently jointly launched the geoCLEF evaluation with Fred Gey and Ray Larson at Berkeley. In recent years, Dr. Sanderson has focused his evaluation research interests to the efficient building of test collections, development of new evaluation measures and the study of query log data and last year worked at the research labs of a major search engine conducting analysis on one of their large log data sets. Research from that work has just been published at the European Conference on Information Retrieval.